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Space Filling Quantizers for Digital Watermarking 
Related Application Data 

This patent application claims the benefit of US Provisional application 
60/256,629, filed December 18, 2000, which is hereby incorporated by reference. 

Technical Field 

The invention relates to steganography, data hiding, and digital watermarking. 

Background and Summary 

Digital watermarking is a process for modifying physical or electronic media to 
embed a machine-readable code into the media. The media may be modified such that 
the embedded code is imperceptible or nearly imperceptible to the user, yet may be 
detected through an automated detection process. Most commonly, digital watermarking 
is applied to media signals such as images, audio signals, and video signals. However, it 
may also be applied to other types of media objects, including documents (e.g., through 
line, word or character shifting), software, multi-dimensional graphics models, and 
surface textures of objects. 

Digital watermarking systems typically have two primary components: an 
encoder that embeds the watermark in a host media signal, and a decoder that detects and 
reads the embedded watermark from a signal suspected of containing a watermark (a 
suspect signal). The encoder embeds a watermark by altering the host media signal. The 
reading component analyzes a suspect signal to detect whether a watermark is present. In 
applications where the watermark encodes information, the reader extracts this 
information from the detected watermark. 

Several particular watermarking techniques have been developed. The reader is 
presumed to be familiar with the literature in this field. Particular techniques for 
embedding and detecting imperceptible watermarks in media signals are detailed in the 
assignee's co-pending application serial number 09/503,881 and US Patent 5,862,260, 
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which are hereby incorporated by reference. A form of watermark embedding referred to 
as Quantization Index Modulation (QIM) is described in WO9960514A1, SYSTEM, 
METHOD, AND PRODUCT FOR INFORMATION EMBEDDING USING AN 
ENSEMBLE OF NON-INTERSECTING EMBEDDING GENERATORS and 
5 corresponding US Patent application 09/206,806, filed December 7, 1998; which are 
hereby incorporated by reference. 

This invention provides a method of QIM digital watermark embedding using 
using space filling quantizers. A QIM watermark embedder uses a space filling quantizer 
to imperceptibly embed an auxiliary message signal into a media signal, such as an 
10 image, audio or video signal. A specific example of the space filling quantizer is a 
IZ hexagonal lattice quantizer. 

VJ A QIM watermark decoder uses a compatible space filling quantizer to decode the 

Is* 

% j auxiliary message signal from a potentially distorted version of the watermarked media 

:£ signal. 

IP to 

; W 15 Brief Description of Drawings 

H Fig. 1 illustrates a block diagram of QIM watermarking. 

Ill 

l r Fig. 2 illustrates quantizers used for dithered uniform scalar QIM watermarking. 

■P Fig. 3 illustrates an extension of scalar dithered QIM to two dimensions. 

}4 Fig. 4 illustrates an example of a sphere packing arrangement to show how to 

20 improve the digital watermark message rate for a given watermark distortion constraint. 

Fig. 5 compares the performance of space filling QIM performance with and 
without distortion compensation. 

Fig. 6 shows performance of QIM watermarking with three different quantizer 

types. 

25 Fig. 7 shows performance of distortion compensated QIM watermarking with 

three different quantizer types. 
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Detailed Description 

INTRODUCTION 

5 Digital watermarking is both an active area of research and a growing source of 

commercial activity. Early research focused upon watermarking as a solution to verify 
ownership of images and other media, but watermarking has since found its way into a 
variety of applications. Some uses for digital watermarking include counterfeit 
prevention, broadcast monitoring, brand protection, and enabling of content on the 
10 internet. 

In these and other applications, digital watermarking allows both the unobtrusive 
embedding of information into media as well as the capability to reliably recover the 
information, possibly after application specific degradation of the watermarked media 
signal (e.g., audio, image, or video). There is a natural tension between requirements of 

1 5 being able to embed a large amount of information, requirements for the embedded 

information to imperceptibly alter the original source, and requirements of being able to 
recover the information after severe distortions to the watermarked source signal. 
Information theory has been applied to the watermarking problem and has been used to 
determine the space of requirements over which we can hope to build real systems. 

20 Chen and Wornell proposed Quantization index modulation (QIM) as an efficient 

method of digital watermarking. QIM is used in the information theoretic proofs of 
watermarking capacity. This capacity can be demonstrated via random coding using a 
sequence of vector quantizers of increasing block length, with both channel capacity and 
optimal rate-distortion performance being reached in the limit of infinite quantizer block 

25 length. It is tempting to investigate the utility of QIM in practical watermarking 

situations as well. It is possible that different considerations should be used in the design 
of quantizers for QIM than for the design of quantizers for compression. Because the 
quantized values are not digitally encoded in QIM, the number of reconstruction values in 
each quantizer is not a design constraint, as it is in the design of a conventional quantizer. 

30 The lack of a rate constraint in QIM suggests that quantizer design for QIM involves 
different considerations than does quantizer design for rate-distortion performance. 
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Lookabaugh has identified three types of advantages of vector quantizers vs. scalar 
quantizers. See T.D. Lookabaugh and R.M. Gray, "High-Resolution Quantization 
Theory and the Vector Quantizer Advantage/' IEEE Trans. On Information Theory 35, 
pp. 1020-1033, Sept. 1989. These advantages are called the space-filling, shape, and 
5 memory advantages. The space-filling advantage is due to the increasing efficiency with 
which polytopes can be used to fill space as the dimension increases. The shape 
advantage involves the marginal distribution of the source signal and the tendency for the 
typical source distribution to be concentrated in a small volume in high dimensional 
space. Finally, the memory advantage exploits dependencies between samples of the 
10 source signal. This document describes digital watermarking methods that exploit the 
space- filling advantage in particular, and provides a heuristic argument that the space- 

p filling advantage alone is sufficient to achieve watermark capacity in some situations. 

1 s QIM performance of some relatively simple quantizers with a space filling gain and with 

! ;P a shape gain are compared and found to support the idea that space-filling gain is 

| y 15 significant for QIM applications of quantizers. 

I , The first section provides some notation and a brief review of QIM. Then the 

III next section investigates the construction of vector quantizers for QIM that have a space- 

> filling gain and no shape or memory gain. Final sections present QIM performance 

results, and the final section presents conclusions. 

3 km 

20 

NOTATION AND REVIEW OF QIM 

The basic watermarking task is to communicate a watermark message Mby 

modifying a host signal X? to produce the watermarked signal . The watermarked 
25 signal then undergoes possible distortions over a watermark signal channel p( \ S?) 
resulting in the received signal Y X N . In some cases it will be helpful to consider the noise 
introduced by the watermark signal channel as additive with variance a 2 . Throughout 
this document, the host signal, watermarked signal, and received signal will be assumed 
to be taken from 9?" , an N-dimensional set of real numbers. An embedding rate R m is 
30 used to embed the message in the host signal; accordingly, the message 
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Me {0,1,2,. . .,2 NRm - 1}. It will be assumed that the distortion introduced by the 
watermark signal channel is independent of the message, and that the host signal is also 
independent of the message. The goal in the design of the watermarking system is to 
minimize the error probability in transmitting the message subject to a constraint D m on 
5 the distortion introduced by the watermarked signal The distortion measure used here 
will be the per letter squared error: 

QIM Watermarking 

5 10 Figure 1 shows how QIM can be used to implement the watermarking task. The 

|«s: 

13 watermark encoder associates a separate quantizer Q with each of the possible 2 NR ™ 



*..5 



messages. It chooses the quantizer Q m corresponding to the input message M and uses 
this quantizer to quantize samples of the host signal, producing the watermarked signal 
01 ^ These samples may be spatial, frequency, time or other transform domain features 

fy 

s 15 of the media signal. After this signal is passed through the channel, the watermark 

fy decoder's task is to estimate the message M based upon the received signal Y X N . This is 

j:;; done by quantizing Y X N using each of the 2 NR m quantizers available to the encoder. The 

r* : corresponding distortion is calculated for each quantizer, and the quantizer resulting in 

? " 

the lowest distortion is assumed to be the one most likely to have been used by the 
20 encoder. Then the decoder chooses the message M , which corresponds to this 

quantizer, as its estimate of the original message M The watermark encoder can be 
defined by 

F(m,x) = Q m (x\ 

and the decoder by 
25 GO0 = arg min D ^QM) 

0<m<2 m >»- 1 

Figure 2 illustrates dithered uniform scalar QIM. The message consists of a 
single bit that causes the encoder to choose between quantizer £) 0? shown at top, and 
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quantizer Q\, shown below. The decoder effectively uses the quantizer shown at the 
bottom of the figure to decide between the two possible message values. 

Distortion 

Chen and Wornell propose a postprocessing technique for QIM that they call 
distortion compensated QM. See B. Chen and G.W. Wornell, "Preprocessed and 
Postprocessed Quantization Index Modulation Methods for Digital Watermarking," in 
Security and Watermarking of Multimedia Contents II, P.W. Wong and E J. Delp, eds., 

Proc. SPIE 3971, pp. 48-59, 2000. They associate a scale factor — with each quantizer 

a 

and use the parameter a to add part of the quantization error back to the watermarked 
source signal: 

F DC (m,x) = Q m (x,A/a) + (l-a)[x-Q m (x,A/a)] 

The parameter a is restricted to [0, 1], ensuring that D(y, Q m (y, A/a)) is 
independent of a . Chen and Wornell showed that the watermark performance can be 
optimized by choosing a so that 

DNR 

a = , 

DNR + \ 

Where DNR= DJa 2 . 



JRM.lmp P0500 12/14/01 



-7- 



EXPRESS MAIL EL593788842US 



DEVELOPMENT OF SPACE-FILLING QUANTIZERS FOR QIM 



10 



15 



Two Dimensions 

Figure 3 shows how dithered uniform scalar QIM can be extended to QM using 
optimal uniform two dimensional quantizers. In the one dimensional case, the Figure 
shows a region of 9? 1 (one-dimensional real number set) which is partitioned into two 
decision regions, one for each possible message value. Each decision region contains a 
code value at its centroid and is a polytope with minimal second moment about its 
centroid (in this case, trivially, an interval). By tiling 9? 1 with the two decision regions 
and codevalues, it is possible to construct the quantizers shown in Figure 2. 

The right side of Figure 3 shows a similar diagram in two dimensions. Here the 
optimal space- filling polytope is the hexagon; correspondingly, a region of ft 2 (two 
dimensional real number set) is partitioned into three hexagons, each with a code value at 
its centroid. Tiling 9? 2 with this region results in a hexagonal lattice; each lattice point is 
assigned to one of three quantizers, and each of the three quantizers themselves have 
code values in a hexagonal sublattice of the original lattice. If code value (J o is chosen 

to lie on the origin, then the lattice of all code values consists of: 

0 V3A _ 



{p:p = Um,m e Z },whereU = 



2A 



20 



where Z is two-dimensional set of integers. 
The lattices of the three quantizer's codevalues are given by 



Qi = {P : P = Vm + o i9 m g Z 2 },whereV = 





V3A~ 




"0" 




' 0 " 




"V3A" 


3A 


3A _ 


,o 0 = 


0 


><>x = 


_2A_ 


,o 2 = 


_ A _ 



25 



Higher Dimensions 

To get an idea of the performance which can be achieved from large dimensional 
quantizers having only a space-filling gain, the previous two dimensional case can be 



*""" 
i? 5 !! 

2 
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extended to an arbitrarily large number of dimensions n. To simplify the analysis, it will 
be convenient to apply some results of high resolution quantization theory, which 
assumes that quantizers have sufficient output levels so that the source probability 
distribution is essentially uniform within each quantization cell. The results of the 
analysis will be valid to the extent that the watermark distortion constraint D m is small in 
comparison to the variance of the watermark source. 

Here it will be assumed that the watermark signal channel is an Additive White 
Gaussian Noise (AWGN) channel with variance a 2 . For large n the channel noise will, 



with high probability, lie within an ^-dimensional sphere of radius 4nc 2 . Thus if the 
g s 10 decision regions of the watermark decoder are spheres of that size centered about the 
13 corresponding quantizer codevalues, the probability of watermark decoding error will 

approach zero with increasing n. Maximizing the watermark message rate for a given 
watermark distortion constraint is equivalent to solving an rc-dimensional sphere packing 
problem similar to a problem that can be used to show the capacity of the Gaussian 
15 channel. 

This is illustrated in Figure 4, in which one decoding sphere for each possible 
message value is packed within a larger sphere. 9T (an n-dimensional space of real 
numbers) can be tiled with this larger sphere to produce the set of all decoding regions 
used by the decoder. Each quantizer of the watermark encoder has quantization cells 
20 which are spheres of the same size as the larger sphere in Figure 4. These quantization 
cells are packed tightly to fill 9T . Each of the quantizers is identical except for an offset 
in 9T corresponding to the location of the corresponding decoding sphere in Figure 4. 
The capacity of this idealized QM is given by the ratio of the volumes of the smaller and 
larger w-dimensional spheres. The larger sphere must have a normalized second moment 
25 about its centroid equal to nD m , the distortion constraint. This is given by 

— ^r 2 = nD m . For large n, the radius of the larger sphere approaches ^nD m . Since 

the volume of an n-dimensional sphere is k n r n , the maximum number of decoding 
spheres that can fit inside the larger sphere is: 
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n'2 *W^. 



This results in an operational watermark capacity for this space-filling QIM of 

q - Ibg^f- bits per sample. If the source X" is iid Gaussian, the watermark capacity 
2 a" 



is given by 



C = ^ logf — + 1 j bits per sample. 



It is interesting to note that if the outer sphere with radius yjn(D m + a 2 ) shown in Figure 



4 is packed with decoding spheres of radius ^Jncr 2 9 exactly enough decoding spheres fit 
13 to reach capacity; a QIM scheme using this larger sphere, however, would not satisfy the 

|*i distortion constraint D m . 



10 

Higher Dimension Distortion Compensated QIM 

The capacity of a higher dimensional distortion compensated QIM can be 
analyzed in a similar manner to the previous case. The larger sphere available for 
1 5 packing of decoding spheres has a radius of 

.j«jk.(p.+<>')ccr: 



a D, 



m 



The distortion compensated watermarked signal will lie within a sphere of radius 



a 2 fiiD m 



r D c = {l-a)r la r S e= D 



centered about each quantizer reconstruction value. Each decoding sphere contains the 
20 sum of the distortion compensated watermark signal and the channel noise. Since these 
two are independent, each decoding sphere has a radius of 



decode 



nD m +n<T -Ana 
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Taking the ratio of the volumes of the two spheres gives a capacity of C = - lo gpr bi ' 

per sample, exactly reaching the theoretical capacity. 

Figure 5 compares the idealized high dimension space-filling QM performance 
with and without distortion compensation. 



PERFORMANCE EXAMPLES 

To investigate how vector quantizers with different types of quantization gain 
perform in a QIM application, QM using three types of quantizers were implemented. 
The first quantizer was the dithered scalar quantization proposed by Chen and Wornell. 
The second was the hexagonal lattice quantizer described above. The final type of 
quantizer simulated for QM was a structured vector quantizer (SVQ). In all cases the 
QM used a blocklength of 32 samples in which either one of two messages (for dithered 
scalar QM and SVQ QM) or one of three messages (for 2-D hexagonal lattice QM) 
was embedded. In the case of 2-D QIM, the same message was embedded in each of 1 6 
pairs of samples in the 32 sample block. 

The message can be used to represent an auxiliary data symbol (e.g., binary, M- 
ary, etc.). Groups of redundant or different symbols may be formed into a digital 
watermark message payload and embedded in the source media signal. To embed such a 
payload, the encoder maps the messages to corresponding samples, or blocks of samples, 
in the source signal. These samples may be spatial, time, frequency, or some other 
transform domain sample of the source signal. 

Examples of multi-dimensional quantizers that can be used for QM 
watermarking include trellis coded quantizers and lattice vector quantizers, such as a 
hexagonal lattice quantizer. These multi-dimensional quantizers enable exploitation of 
the shape filling gain described above. 

Structured Vector Quantizers and QIM 

SVQ exhibits no space filling gain, but does take advantage of the available shape 
gain. It is able to achieve the performance of an entropy coded quantizer, but at a fixed 
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bit rate. The first step in designing an rc-dimensional SVQ is to first design an N level 
entropy coded scalar quantizer and to calculate the codeword lengths corresponding to 
each of the quantizer's levels. Then an n-dimensional product codebook, containing If 
codevectors of length n is formed. Each vector in the product codebook is assigned a 
length corresponding to the sum of the lengths of the corresponding scalar quantizer 
codeword outputs. Then a threshold T ^nU (X), where H(X) is the entropy of the 
source is chosen. Finally each codevector with a length greater than T is then removed 
from the product codebook. Source vectors of length n are then quantized to the closest 
codevector in the remaining codebook. The number of bits required to represent each 
codevector is approximately T, which is less than nlogiN. As block length increases, the 
performance of the SVQ approaches the distortion and rate performance of the original 
entropy coded scalar quantizer. 

For QIM, two SVQs were designed, both with block lengths of 32 samples. The 
scalar quantizers used as the basis of these two SVQs were dithered uniform scalar 
quantizers. Logarithms of scalar quantizer probabilities were used instead of codeword 
lengths. 

Results 

Since the message size is different for the 2-D hexagonal lattice QIM than for the 
other two types of QIM, they cannot be directly compared by using the resulting 
probabilities of decoding error. Instead, the delivered channel capacities are derived from 
the decoding error probabilities and compared. The channel capacities are given by: 

Comer = 1 ■ 0 + Kr W err ) + C " Kr ) logO - 

Csvo = 1 .0 + P err \0g{P err ) + (1 - P err ) logO - P err \ 

c HEX = iog(3)+/^tog(^/2)+a-oioga-o 

Figure 6 shows the results for a unit variance Gaussian source with AWGN of 
22dB. The hexagonal lattice based QIM performed the best, followed by the dithered 
scalar QIM, and the SVQ QIM had the poorest performance. 
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The loss in performance between the dithered scalar quantizer and SVQ is most 
likely due to a slight decline in the distortion characteristics of the SVQ, when compared 
with the original scalar quantizer it was designed from. In a compression application, this 
decline is more than compensated for by the lower rate requirement of the SVQ 
5 compared with the scalar quantizer. Since encoding rate is not important for QIM, this 
advantage does not accrue to the SVQ QIM. 

The hexagonal lattice based QIM performed significantly better than the dithered 
scalar quantizer; in most cases the capacity of the hexagonal lattice QIM was 
approximately twice the capacity of the dithered scalar QIM. 

10 

CONCLUDING REMARKS 

13 The results of the previous section have shown that it is possible to improve QIM 

J.i performance by using a quantizer which provides a space- filling gain; the results have 

;F also shown that quantizers which provide a shape gain do not necessarily improve QIM 

15 performance. The capacity argument presented above implies that, asymptotically in 
I Iu block length and quantizer resolution, space filling gain alone may be enough to achieve 

I II watermarking capacity in distortion compensated QIM. 

*g Having described and illustrated the principles of the technology with reference to 

]t specific implementations in the attached paper and documents incorporated by reference, 

20 it will be recognized that the technology can be implemented in many other, different, 
forms. To provide a comprehensive disclosure without unduly lengthening the 
specification, applicants incorporate by reference the patents and patent applications 
referenced above. 

The methods, processes, and systems described above may be implemented in 
25 hardware, software or a combination of hardware and software. For example, the 

auxiliary data encoding processes may be implemented in a programmable computer or a 
special purpose digital circuit. Similarly, auxiliary data decoding may be implemented in 
software, firmware, hardware, or combinations of software, firmware and hardware. The 
methods and processes described above may be implemented in programs executed from 
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a system's memory (a computer readable medium, such as an electronic, optical or 
magnetic storage device). 

The particular combinations of elements and features in the above-detailed 
embodiments are exemplary only; the interchanging and substitution of these teachings 
with other teachings in this and the incorporated-by-reference patents/applications are 
also contemplated. 



